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ABSTRACT 

A review of selected literature on 
education programs provides a background tor the a 
of the conditions essential to open education and 
its effectiveness. Other factors mentioned are: (1 
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Open fducalion: A Challoncje to rvuluators 



Susan S. Stodoli^ky 
UnivcTsUy of Chicayo 

The purpose of this paper is to review selected literature 
on the evaluation of open education programs. It seems imporLant, 
as a first btep, to clarify what class of educational environment 
is being discussed when people refer to open education. What is the 
nature of the beast? 

It is not really easy to del inn t the meaning of open education, 
but there have been a number of systematic attempts to define the 
term and also to empirically describe a classroom which is more open 
than traditional* the way the teacher might behave, and the sort" of 
child behavior one might see. Some of the most complete and help- 
ful discussions which attempt to define characteristics of open or 
informal classrooms are those by: Barth (2), Bussis and Chitte.iden 
(6); Featherstone (12,13), Flurry (14); Gross and Gross (17); 

Katz (24); Rogers (28); and Weber (40). Although these authors 
vary somewhat in the approaches they fcakfij they tend to agree on the 
central characteristics of open classrooms. They recognize that cpen 
classrooms in the United States have been inspired by thie informal 
methods of British teachers (8). 

Certain basic assumptions about the way children learn are made 
by open educators. They accept the developmental view that, children 
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learn throunli direct experience mi nro curious and luotivaUid to 
loarn. Thoy accept thn idoci tfint children differ in tho wciys thoy 
learn and the times at which various aspects of development occur. 
Open educators believe that children have interests which are expressed 
in their activity and play and that these interests should gGnorally 
be respected and nurtured in the school. The developmental perspective 
adopted by open educators is based on the psychological theories 
and research of Susan Isaacs (21). Nathan Isaacs (20) and Jean Piagot. 

Beatrice and Ronald Gross (17) present a list of principles 
which seem to reflect the views in the literature I have seen on 
open education. They say: 

There are four operating principles of the open c' ass- 
room. First, the room itself is decentralized: an open, 
flexible space divided into functional areas, rather than 
one fixed homoqeneous unit. Second, the children are • 
free for much of the time to explore this room, individually 
or in groups, and to choose their own activities. Third, 
the environment is rich in learning resources, including 
plenty of concrete materials, as well as books and other 
media. Fourth, the teacher and her aids work most of the 
time with individual children or two or three, hardly 
ever presenting the same material to the class as a whole 
(p. 10). 

The notion of the "integrated day" is an important one in 
the British literature on informal education. The core idea is that 
subject matters and contents of learning are not dealt with singly 
but that the child's behavior and activity can lead to the use of 
various subject matters in a given activity. Learning through the 
development of an interest in something, rather tlian by going from 
page 10 to page 11, is favored by many practitioners of informal methods. 
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Thus, the school dny is selduiu divided inl;o blotk;: of time for 
certain subject matters, althou'jh there iiioy br:^ divisions of certain 
types. 

It might be worth emphasiziny that interactions between the 
teacher and the children and among the children are a salient feature 
of open classrooms. Certain proponents of this general approach 
emphasize the development of cooperative learning and group learning 
more than others (for example, the Bank Street model of open education 
stresses child-child interactions). Virtually all open educators* 
however, consider interpersonal interactions to be a significant 
feature of the informal school experience. 

Most authors describing open classrooms and the philosophy of 
open education comment on the very demanding role of the teacher in 
this approach to education. In order to have an educationally satis- 
factory open classroom, the teacher must be able to prepare materials 
which can be used by a variety of children, he/she must be able to 
extend and help develop children's interests, and must be able to 
tolerate more busyness and noise than one encounters in the traditional 
classroom. The teacher must be an astute observer of the children 
and must in some fashion keep track of their work so that she can 
decide when and if to intervene in the day of a child. It is usually 
?rireed that it takes at least two years for a teacher to feel "on top 
of" the demands of the teacher role and that an open classroom teacher 
must still expect to work harder than a traditional teacher (see Andreae, 
1). 
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It would bci easy to (jct tlio imi^n'j\5ion Ui.it with clvilih-(.'n 
following tlicir own inloicsi Uu' bc^t open classroum is one in 
whicli the teacher intervenes least. This pur/itiim Ijds been taken 
by some but seems to reflect a ba'Jic inisuncienitandinQ of informal 
principles. According to Berlak and his colleagues (3), open 
educators do not believe that a child will always pursue the most 
educationally beneficial patli on his own. They believe the teacher 
has an important responsibility to set standards for work and 
conduct, to direct, extend and develop student interests, and at 
times to impose certain requirements on children. Thus, although 
there is a good deal of confusion on this point, open educators 
are concerned with the development of academic skills and standards. 
They do not see, however, the necessity for the development of these 
skills on a fixed timetable and in a fixed mode. Furthermore, they 
do not believe that academic skills are the sine qua non of the 
elementary school. 

I trust this discussion adequately conveys the general character- 
istics of the schooling referred to Yn' the open education literature. 
One of the problems, to which we will return, is that the term open 
education has been used to refer to many varieties of innovation 
other than the ones just described. In reviewing the existing research 
on open classrooms, an important first step is to clarify the actual 
nature of the program which is being studied. Frequently the program 
being studied lacks some of the critical features of open education as 
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I have described it here. For oxaiivlo, lUew) h.io been soino con^triic- 
tion of open plan schools, schoolr, which have open spaces that can 
ba flexibly used. Those open plan faciliticb uro suitable for open 
education programs, but can alr.o house a variety of other programs. 
There are studies which have looked at open plan schools that do not 
house open education program*;. Analogously, many individualized 
instruction programs are being instituted in schools around the 
country, but, individualization alone docs not signify an open 
education program. 

For purposes of this paper, I will attempt to base my discussion 
on the conditions presented in the literature as rather essential to 
an open education program. Open educators are not doctrinaire and 
do not have a single prototype to implement, but they do seem to 
have an essential approach toward which they feel a teacher should 
be oriented. 

If we understand generally v4iat is meant by open education, we 
can next move to the questions of evaluating such programs, I will 
attempt to distinguish two major points of view about the way to 
evaluate open programs and will review the limited amount of literature 
which is available, I will also try to detail some of the major 
problems to be overcome in this area and will suggest some possible 
strategies for arriving at useful solutions. 
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The Clonsic Viov/ J?J'_LC9iO'A'll l-.V^!.^ 

Thfi existonco of open oducatioii |Jtof.iraiiis , us woll as the problciii 
of evaluatiiuj pr'iSchool prog.aiiis of diverse coinpU?xion (35) has contributed to 
the development of new ideas about evaluation. The classic view of 
evaluation focuses on the collection of evidence about a program's 
effectiveness in achievimj certain objectives or rjoals. Classically 
designed evaluations are studies of outcomes. With the emergence of 
large-scale projects v^hich require evaluation data, many iitudius 
have been conducted which attempt to compare one program with another 
or one program with a "control group". 

The classic evaluation paradigm for evaluating open education 
programs poses many problemsi yet the approach has a valid place 
and deserves further effort. 

In the application of tlie classic evaluation paradigm to open 
education programs, an important step (often omitted) is a careful 
consideration of the objectives of open education. Four approaches 
to relate outcome measures to objectives are possible. In my opinion, 
an evaluation of an^ educational program should include evidence about 
the achievement of the intended outcomes of the program. At times, 
an evaluation study will include data which measures atrributes thought 
to be of value whether the program sponsor has specified them or not. 
These atrributes may bu considered valuable on an a p riori basis and are 
not necessarily aims of program itself. If we had consensus about a set 
of desirable outcomes which should result from educational programs 
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for children, we could utilise liiKuiruirob of" tiiat act of outcrw'S to 

assess all pronrams. but I do not l:hinl< wo aro in that position. In 

conducting comparative evaluation studios, ono iiii<iht include measures 

of attributes which ore a collciction of tho intended outcomes of diverse 

programs and thus make some measures probably more appropriate for one 

program than for the other. Last, the evaluator may attempt to measure 

some; attributes which miuht be unintended outcomes of a prooram.. 

Douglas Pidgeon (26) very ably expresses the classic point of 

view with respect to the evaluation of informal education. He says: 

With all the extensive changes. now tai;inci place, 
it Is clear that only an evaluation specifically 
designed to discover the extent to which a vari- 
ety of objectives is being met is likely to pro- 
duce the kind of evidence that will satisfy pro- 
tagonists and antaaonists alike. Such a study 
would have to be carried out on a carefully selec- 
ted sample of schools which represented the best 
and the worst of both the traditional and newer 
informal approaches to teaching and learning. 
The aims and objectives of the sample schools 
should over not only those aspects of education 
which are given more emphasis with the informal 
approach, but also those which the critics of 
change hold in high esteem (pp. 19-20). 

There are some examples in the literature of studies which have 
selected outcomes to be measured in the four ways I have mentioned. 
Until very recently, the most common practice was the use of measures 
which seemed valuable in their own right and which did not necessarily 
reflect the objectives of highest priority for a given program or 
even objectives being attempted. Most of the Head Start evaluations were 
of this sort, with the ubiquitous intelligence test serving as a major 
component. I have reviewed elsewhere the neneral nature of tho findings 
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Of preschool ovaluntion pcoui-.^.m:", (r?1» 34). Tv/n f'oinlr. I'lorit roi It? ra- 
tion hore hecciuse they dr*.' 0(.,u.)l].\ fi*'*""'^^''^- tn thi! work in open oducfttion. 
First, the pattern of reL.uIts in the oxibtinfi 1 i l.<.Tature nener(^ny 
show more successful cutcoiwii for litruttured tfinr. for unstructured 
procirams. Second, this apnears to bo the case because the iwasui^es 
used in the studies tend to r^f^flect the objectives of the structured 
programs better than the unstructurf^d ones. It is also thp case 
because the structured prociraiiis are more iiomofjffneous with respect to 
the experiences of the children. Thus, the structured programs meet 
the important intthodolonical standard of beiny a "treatment" to be 
evaluated, whereas more unstructured pronrams tend to be much less 
uniform. In unstructured or open situations, children enuage in 
diverse behaviors. While diversity is a core commitment for open 
educators, it cjreatly complicates any attempt to study the effects of 
schooling. 

Why has there been so little assessment of the intendend out- 
comes of open education? ; believe one central difficulty has boon 
the lack of procedures for assessing the i<inds of behaviors of interest 
to open educators. 

It should be clear from the earlier discussion of open education 
that the objectives of highest priority lie outside the easily measur- 
ed doti;uin of skill learninq. In fact, many of the central objpctives 
of open education arc net in the connitivo domain at all. L-ven for 
those educators who have reconni^ed the qoals of open classrooms as 
valid, a serious problem still remains in locating or developing 
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or dovolopiiui mothods for studyiiKi sucli chariicti.'ri sties in childrt'ii. 

Jennifer Androae (1) cnpturrjs inany koy ohjpctivcs in tlio follov/lny 

discussion of evaluation. 

The test of true leorninn is in the. .ibility U) use 
appropriate thinkinn fikiils to ii"^f.'t such siluution. 
Standcirdized achiovcnujnt tests do not sliow Ifris; nor 
do they show initiative; responsibility: ahility to 
relate to others j or level of intoH It'ont thinkinti; 
or attitudes tcwdird lenroinu; or cdrryovf?r homo; 
or whore and what the Ocips <"\r? in a child's undor- 
standinq, ability to bolvo problems, use of 
imagination, creativenoss , and innenuity; or aware- 
ness or sensitivity to a child's interest and 
learning style.. At the present time, thorf-^ are 
no standardized tests to measure these vitally 
improtant factors, and yet without this Inlorina- 
tion, the picture presented of a child is a shallow 
■and nearly meaningless one (p. 57). 

There huve been some attempts to get at certain aspects of self 
concept, attitudes toward schooling, achievement motivation and locus 
of control (10, 18, 26, 28, 29, 30, 37). Methodologically, the quality 
of these studies is varied and the ranne of instruments narrow. 
Lac k of Instruments in the Affective Dom ain 

While virtually all workers in the affective domain have lamented 
the lack of adequate instrumentation, there have been very few projects 
which have focused on the development of psychometrlcally reliable 
and valid instruments of socioemotional functioning, curiosity, 
initiative, interest and the like. A recent book by Oeborah K. Walker 
(38) does provide a very helpful listing and critique of existing 
socioeii'otional measures for preschool and kindergarten children. 
Presumably, tlio instrument construction problem is somewhat less 
difficult when one deals with children of elementary school age. V.'here 



11 



BEST COPX MIttiABli 



it; ir> oasier to int,orvii"?w younci chilJiv'n and yi>t ii:o:iii inqf ul v(;si.on':.os , 

It becomes possible to consider the use of qroup measures and naper and 
pencil devices. 

Certainly onv. reason for tho lack of instruments in tin's broad 
area has betm i\ feeble cornnri Imcnt to the importance of such objectives, 
coupled with the vr?ry surinus problcnni of the fiiirntJss of evaludtinc) • 
children vn'th ronpect to attitudes, values, and other personal attri- 
butes. Our American values of individuality, privacy, lacic of indoctri- 
nation and so on, have led many school pnople to sfiy away from the 
collection of evidence about affective behaviors. While concerns 
about values should not be glossed over lightly, the lt'cl< of evidence 
about affective and attitudinal development seriously limits our 
ability to adequately judge the consequences of educational experiences. 
Furthermore, one can collect such information without necessarily 
using it in any potentially punitive context. There is a difference 
between information to be used for student evaluation and information 
to be used for pronrain evaluation. While overlap in these two types 
of data would be efficient, it may not always be possible. At least 
it would 1)0 helpful to have data on socioemotional atid affective 
variables for program evaluation. 

Few sustained efforts at affective instrument construction for 
school children have ben launched, and those instruments which have 
been developed often have lov validity and cover but a fraction of 
the richness of behavior in which we are interested. I^lhile continued 
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and renewed offortfi hould bo supportod, I have noim' n-curri^nt ticiuhts 

about tho potential of such ef forts. The use of ci U(!rion-ror(.M-cn(.('d 

rather than norm-rcrerencod instruntonts may be.heliiful (lb, 37). 

Yet my vMork with children in different kinds of clussrvooms makes me 

question whether the psychonu.-tric approach will prove rolevant to 

many of the constructs which interest us. I find it somewhat consol- 

ina that other colleagues involved in tho evaluation of educational 

programs have voiced similar doubts. Herb Zimiles (42), in a discussion 

of personality measurement, suggests that Allport's idiographic view 

may have merited more support by psychologists than it received. 

. . . different traits are differentially salient for 
different children (or adults). Across-the-hoord 
measurement of a particular trait generates a hodg^?- 
podge of data. The data gathered from thosp for v/hom 
the trait is salient may be quite telling, but a laro-^ 
portion of the data obtained from the rest of the sample 
may have no functional significance (p. 5). 

Recently, I have been attempting to understand the choices 

children made in classrooms which were moving toward an open philosophy. 

I used a series of psychometric instruments which I thought might 

help differentiate children in conjunction with observations of 

children's classroom behavior. I was interested in finding out what 

sorts of children select what sorts of activities and erhibit different 

styles in an open setting. There appear to be relationships between 

some of the characteristics I measured (intelligence, cognitive style, 

locus of control, and associational fluency) and certain aspects of 

the child's functioning in the classroom--for example, t!ie extent to 

which he pursues activities alone. However, various aspects of child 
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fundi oninu srciii tn rvlcrlo to soinowiwit. iniloii.'Midciil clui.tin'irifjS of 
child variables (and, of cowrsa, som rolal.o lo nruic of tJic vuridhlus) 
furthermoro, our clinical iniprousions of tho children load us to vary 
different hypotheses about the correlates and otiolony of similar 
behavior patterns in different children. 

What I am trying to illustrate here is the perennial tension be- 
tween the recognition of individuality and the attempt of measurement 
to provide comparability of data under standard conditions. While 
both endeavors are necessary, we may be least successful in applying 
the psychometric approach in the socioemotional area. Nor am I 
certain that a quantitative view is appruoriate for many aspects of 
personality and affect. Many of the constructs which interest open 
educators are more qualitative in nature and depend upon certain 
conditions for their expression. For example. I am not sure that it 
is meaningful to ask whether a child has a more positive or negative 
self concept than another child rather than knowing the natu re of his 
views of himself. Similarly a construct like initiative seems to come 
in many behavioral forms which are difficult to assess in a uniform 
manner across children. 

An extremely insightful article by Edna Shapiro (31) contains 
a discussion of the relative utility of tests and classroom observa- 
tions as sources of data. She says: 

The rationale of the test, ... is that each child Is 
removed from the classroom and treated equivalently , 
and differences in response arc pro?,umed to indicate 
differences in what h.is been taken in, made one's own, 
that survives the shift to a different situation. 
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. . , This rpquirfir. rocntv. idniMtion of tlio y'olr» of 
cla«>srooin dcU.a, irulividUf^l l.^sl; si LunLion daU, and IIk.' 
relation botvoon thm. ]f wo ininiiiri;'^ tho iniporlfinco 
of the child's hchavior in tho clasr.room buctiii:.(» it is 
influenced by situotinnal vdrinblof., do wo not. fifwo to 
apply the sane locjic to the child's n?spon'.es in the tetit 
situation, which is also influenced by situational 
variables (pp. 633-34)? 

Shapiro does not consider classroom data and test data to be 
equally useful for all evaluation purposes, but she seriously 
questions whether the sort of test data we are able to obtain will be 
any more useful for program evaluation than classroom observations 
would be. She eventually argues that evaluation should play its chief 
role as a contributor to program development and as such would justify 
more flexible uses of observations and experimental instruments. V.'hile 
I agree vnth this general point, I am also sympathetic to policy makers 
who want evaluation data in order to make policy decisions about the 
effectiveness of programs. Perhaps a consideration of some suggestions 
and studies made by proponents of open education and evaluators who 
do not hold the classic view will assist us to reach some useful 
conclusions about the best evaluation strategies. 
Alternative Views of Evaluation 

A number of persons involved in the field of evaluation have 
questioned the appropriateness of measuring outcomes as the basis for 
judging program effectiveness. Many open educators advocate a process 
of documentation of child growth and development based on children's 
work and actual classroom behavior (see Carini, 7, Chittenden and 
Bussis, 9, and Dean, 10). Some helpful examples are presented in a 
recent article by Hawes (18) which reports the work of Huth Aldr.ch 
at Marcy School, Lillian Weber, Mary Stitt and others. A number of 
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evaluators, includincj fnyscir (31 , 33, 30), hav.? also roconii'tciulnd that 
the classroom Inihavior of childivn he f.ofir.idc red valid and ii ;;>urtatit 
evidence about a program. 

There are several positive aspects to this nenoral stratociy. 
First, it is important to better understand the actual classroom 
experiences and processes of children in diverse educational settiricis. 
Data which details the salient features of various classroom environ- 
ments and of different children's experiences in different classrooms 
can be used for a number of highly significant purposes. 

Most germane here is the use of such descriptive data to make 

judgments about the quality of an educational program. Zimiles (42) 

persuasively argues for: 

. . . systematic and comprehensive evaluation of the 
child's school environment, to be followed by a 
theoretical analysis of the potential impact of his 
school experience. 

... It is a paradox that we have the responsibility 
and the capacity comprehensively to describe and 
record the essential character of an educationdl 
program, yet do not do so; and at the same time, 
we do not know how to assess the impact of a co;;iple 
set of experiences on the psychological functioning 
of a developing child, yet we persist in trying to 
do so (pp. 7-8), 

Zimiles believes that we can look at educational environments and 
make value judgments about them. Such judgments would be based on 
our theories about the educational conditions which lead to certain 
desirable ends. Zimiles stresses the necessity for a conceptual 
framework which would allow us to make connections between what we 
see in classrooms and our ideas about the "potential impact" of these 
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conditions on the child. I believe that heads of English schools 
and Inspectors who routinely evaluate and advise British teachers 
implicitly use this strategy. 

A elated point of view is presented by Chittenden, Bussls* 
and Amarel at ETS (see Hawes, 18). They suggest that the quality 
of the classroom experience should form the basis for evaluation of 
open classrooms. They would evaluate diverse aspects of both student 
and teacher behavior and products as well as aspects of the physical 
environment of a classroom. In contrast to conventional evaluation 
which judges student responses to some uniform condition, they are 
trying to develop standards for judging diverse student productions 
and activities. The ETS approach to evaluation focuses on standards 
of quality regarding the "process, content and context of the 
child's learning" (Hawes, 18). 

Both the ETS group and Zimiles endorse the need for evaluation 
evidence to judge programs . They are searching for new ways of 
producing evidence which will in some sense be objective and comparable 
across educational environments. I share their concern about data 
which will allow one to make judgments about different educational 
experiences, but I believe that program evaluation per se may be less 
important for the educational enterprise than evaluation which will 
feed back into student learning and studies which will begin to 
explicate the nature of the empirical relationships between various 
educational experiences, child characteristics, and child outcomes. 
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It inoy not ho nccossar^y to pit one.' type of work «9ciinr>t the 
other. Yet I believe larcio \ca\Q classic ovaluatiori'j have nb'.urbcd 
too much of our limited ^sources without adequately answerinn the 
key questions. 

Open education presents a real challenge to evaluators and educa- 
tional researchers because it broadens the scope of educational environ 
ments and experiences to be understood. We have begun to develop 
methods for systematically describing some of the relevant features 
of classroom ?^ructure and process. Promising observational procedures 
have been developed in response to the need for systematic descriptions 
of educational environments (5, 16, 23, 24, 32, 34). 

There appear to be some promising starts on the development 
of an empirical base for understanding how educational environments 
differ (4, 15, 32, 34). Some possible consequences of the educational 
differences are also being studied. For example, in a recent paper 
using Follow Through data, Stallings (33) related differences in class- 
room conditions to differences in children's classroom behaviors and 
to tested outcomes. With respect to open classrooms, she says: 

In the more open, interdisciplinary classrooms, where 
a wide variety of activities are occuring, a wide 
variety of materials are available, children can 
select their own groupings part of the time, and can 
engage in activities without adults, children have 
higher scores on the Raven's perceptual problem- 
solving test. They are also absent less often, 
and they take more responsibility for their success 
as measured on the Intellectual Responsibility Scale. 
They are more independent, cooperate more often, and 
ask more questions (p. 13). 
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Replication «nd oxtension of tiri irt of work situuld .1 lri(|h 
priority. 

My students and I have baan attcinptino to contribute in this 
area by intensive small-scale studies which followed individual 
children over time in free-choice classrooms. Wc have been able to 
use samplings of children's behavior to define variations in the 
children's experiences which occur in a given classroom. The5;e 
variations in activity have, in turn, been related very strongly to 
differences in certain outcome measures (see Karl son, 22). Since 
students clearly do pursue unique paths within an open classroom, it 
seems educationally important to know more about the types of choices 
they make and the types of behaviors they exhibit. We hopo to unearth 
possible correlates of choices and styles both in terms of individual 
differences in the students and in relation to the situational 
determinants of choice. Welch (41) has been able to show at the 
nursery level that cognitive style differences are strongly predictive 
of differences in the free play behavior of children, particularly 
with respect to the pacing of activity and patterns of attention. Py 
most recent work with seven- and eight-year-olds also seems to be 
revealing that some systematic variation in child behavior in open 
environments is related to measurable individual differences. 

In addition to process-oriented studies, we should shift from 
evaluations of programs themselves to evaluations of student Icarninn 
and development for feedback purposes. Teachers continuously observe 
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students and analyze their activity and products in an effort to 
ascertain the best strategies for facilitatinq educational development. 
Systematic and long-terpi efforts of this type v.'ould contribute both 
to the immediate process of education and to our undorstandinn of the 
consequences of educational programs. Obviously, evaluation directed 
toward a feedback or formative function is designed primarily, if not 
exclusively, in terms of the intended goals and objectives of a 
given program. 
Recomniendations 

In order to improve the classic approach to evaluation of open 
education, I would recomm'i>'r( the following actions: 

1) Empirical description of the nature of the educational 
environment under study should be a necessary step in all evaluations. 
It is essential to verify that a program is actually being implemented 
and to record the salient features oiF the program. 

2) Where technically feasible, students should at least be 
measured with respect to the high priority objectives that the program 
itself endorses. Where tests are not available for this purpose, 
consideration should be given to using classroom observations and 
other si tuationally complex methods for assessment. 

3) Long-term studies of children and environments are essential. 
The objectives of open educators are not expected to be easily 
obtainable. Only longitudinal evidence can effectively address the 
question of outcomes for open education (I believe a similar argument 
could be made with respect to many educational programs.) 

4) Instrument construction, particularly in the socioemo- 
tional and affective domains, should be attempted with validity as 
the key criterion in judging the utility of new instruments. 

5) Rating and judgment methods should be further explored as 
possible alternatives to test situations. 

6) The use of children's own reports through interviews, logs, 
and other devices should be considered valid evidence about program 
process as v^ell as student development. Similarly, the views of 
parents and teachers can be crucial ingredients in an evaluation plan. 
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Outside the domain of classic evaluation, I would recoimiond 
the followinf): 

1) Systematic efforts should be made to col lor I, work, samples and 
behavior observations of children in open classrooms. Where possible, 
the standard design considerations should be employed to avoid bias, 
lack of randomization, and so on. 

2) Efforts should be made to develop and explicate a theory which 
would permit inferences about the potentinl impact of an educational 
environment on child growth and development. 

3) /\s the above theory is developed, empirical validation, 
wherever feasible, should be attempted. 

4) "Ihe collection of case studies on children over long periods 
of time would provide helpful information for making judgments both 
about the quality of an educational experience and its possible 
consequences. Such materials might also add to our understanding 

of human variation in development. When possible, such case studies 
should contain information about the actual experiences of the child 
in the educational setting so that we may better understand his 
educational "treatment." 

5) Comparative studies of process in classrooms, preferably 

at a small-scale, Intensive, rather than large-scale, extensive level, 
should be supported. To the extent that open education presents 
important variants in educational environments, understanding the 
components of open programs with respect to other programs should 
clarify the different contributions to be made by these varieties. 

During the writing of this paper, I have felt discomfort with the 
task. I strongly feel the lack of highly specific, concrete suggestions 
in what I have written. Hopefully the more general considerations will 
prove stimulating to those more intimately involved with the day-to-day 
process of education in the classroom. I have tried to provide through 
references a key to those helpful methods and ideas which are currently 
available in print. Sadly, the field has not moved very far yet. Never- 
theless, the challenge of open education represents an important and 
sustained impetus for change--which should both please and perplex us. 
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